Thus far, our MLMs have grouped observations within essentially nominal categories – even when we use numbers, we’re treating these categories as unordered and discrete. A participant id is just a label for a unique “thing”. In these models, partial pooling does double duty – it improves accuracy by borrowing information from other groups but estimates variation across groups.
But consider other types of grouping that are not so distinct – McElreath shows examples using spatial distance, but also consider time or age. Individuals of the same age share some exposure (cultural trends, political and historical events, even climate); they also share exposure with people of similar ages. And even though we assign labels to generations, guessing the similarity between two people is probably better done by the differences in their ages rather than whether they share a generational label.
In this case, it wouldn’t make as much sense to fit a separate intercept for individuals of the same age, because the model would borrow equally from all other groups. Instead, we would rather the model borrow information in proportion to the closeness of other groups (e.g., intercepts for 27-year-olds should be more informed by data from 28- and 26-year-olds than 67-year-olds).
The general approach to this is known as GAUSSIAN PROCESS REGRESSION.
research questions
Gaussian processes can be useful for answering questions like:
How do effects vary smoothly across continuous variables?
What is the functional relationship between predictors and outcomes when the form is unknown?
How do temporal or spatial factors influence outcomes?
What are the similarities between observations based on multiple dimensions?
How can we make predictions with appropriate uncertainty for new, unseen values?
How do multiple signals interact in complex, non-linear ways?
What’s the underlying pattern when we have noisy measurements?
A GP is a stochastic process, which describes the relation between one or more predictors \(x = (x_1, ..., x_d)\) and a response \(f(x)\), where \(d\) is the number of predictors. A GP is the generalization of the multivariate normal distribution to an infinite number of dimensions. Thus, it can be interpreted as a prior over functions. The values of \(f()\) at any finite set of locations are jointly multivariate normal, with a covariance matrix defined by the covariance kernal \(k_p (x_i,x_j)\), where \(p\) is the vector of parameters of the GP:
The smoothness and general behavior of the function \(f\) depends only on the choice of the covariance kernel.
kernals
The smoothness and general behavior of the function \(f\) depends only on the choice of the covariance kernel. In plain English, a kernel is like a “relationship measurer” between any two points in your data. It answers the question: “If I know the value at point A, how much does that tell me about the value at point B?”
The kernel determines how the influence spreads across your data. If two points are close together according to the kernel, their values will be similar. If two points are far apart, their values will be uncorrelated (but not necessarily far apart).
Different kernels create different patterns of relationships. For example:
Some kernels make smooth, gentle transitions between points
Others allow for more abrupt changes
Some might say that periodic patterns exist (like seasonal effects)
The mathematical formula of the kernel determines exactly how this similarity decays with distance, time, or whatever dimension you’re working with.
Currently there are four available kernels in brms (in order from smoothest to roughest):
With magnitude \(\sigma\) and length scale \(l\), the exponentiated quadratic kernel is:
\[
k(x_i, x_j) = \sigma^2\text{exp}(-\frac{|x_i-x_j|^2}{2l^2})
\] * Think of it as creating very gentle hills and valleys - no sudden jumps or sharp corners * Influence between points drops off quickly (like a bell curve) * If you know the value at age 25, it gives you strong information about age 24 and 26, moderate information about age 22 and 28, and very little about age 15 or 35 * Best for processes you believe are inherently smooth and continuous * Example: Physical growth patterns in children might follow this kind of smooth curve
Matern 3/2
This creates somewhat rougher patterns than the exponentiated quadratic.
With magnitude \(\sigma\) and length scale \(l\), the Matern 3/2 kernel is:
Allows for more flexibility than exp_quad while still maintaining good continuity
Example: Environmental factors like temperature across a landscape might follow this pattern, with generally smooth transitions but allowing for some local variations
exponential
This is the roughest of the four kernels.
With magnitude \(\sigma\) and length scale \(l\), the exponential kernel is:
The influence between points drops off very quickly with distance
Neighboring points still have relationship, but it weakens more rapidly than with other kernels
Good for processes where you expect more abrupt changes or less smoothness
Example: Opinion shifts across different age groups during times of rapid social change might follow this pattern, with sharper transitions between cohorts
example: data
Data come from a 26-wave (every 2 weeks) study of political attitudes (Brandt et al., 2021).
Should federal spending on defense be increased, decreased, or kept the same?
1 - Greatly decrease
7 - Greatly increase
Code
d %>%count(def) %>%mutate(col =ifelse(def >7, "1","2"),def =factor(def, levels=c(1:9),labels =c("1\nGreatly decrease", "2", "3", "4\nKeep the same", "5", "6", "7\nGreatly increase", "8\n Don't Know", "9\n Haven't thought"))) %>%ggplot(aes(x=def, y=n)) +geom_col(aes(fill=col)) +scale_x_discrete(labels =label_wrap(10)) +labs(title ="Should federal spending on defense be changed?",x=NULL,y ="count") +theme(legend.position ="none")
some data cleaning
Let’s remove those values on the variable that are not part of our Likert scale.
d =filter(d, def <=7)
We’ll be adding age at baseline to the model. Age was only collected during wave 1, so we’ll simply carry that through all the survey responses. There’s a few ways to do that. I’ll show you how to use the fill function in tidyverse. We’ll also remove anyone who didn’t report an age.
d = d %>%arrange(id, wave) %>%with_groups( id,~fill(., age, .direction="down") ) %>%filter(!is.na(age))
model 1: intercept only
Let’s say we’re interested in studying how time and age impact the responses to this question. We’ll build up our models from most simple to most complex.
\[
k(x_i, x_j) = \sigma^2\text{exp}(-\frac{|x_i-x_j|^2}{2l^2})
\] The value \(|x_i-x_j|^2\) is the absolute squared distance between two values. That means, the two values estimated in our model are \(\sigma\) and \(l\). What do each of these mean?
Let’s start with \(l\): The value of \(1/(2\times l^2)\) is equal to \(\rho\). So we can rewrite the equation as
\[
k(x_i, x_j) = \sigma^2\text{exp}(-\rho|x_i-x_j|^2)
\] In other words, the covariance between two values is declines exponentially with the squared distance between them.
The remaining piece, \(\sigma^2\) is the maximum covariance between any two values.
I used an invgamma distribution for as my prior for \(l\) and an expoential distribution as the prior for \(\sigma\). Here’s what that looks like:
